Focus scanning apparatus recording color

ABSTRACT

Disclosed are a scanner system and a method for recording surface geometry and surface color of an object where both surface geometry information and surface color information for a block of the image sensor pixels at least partly from one 2D image recorded by the color image sensor. A particular application is within dentistry, particularly for intraoral scanning.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.18/183,358, filed on Mar. 14, 2023, which is a continuation of U.S.application Ser. No. 16/921,414, filed on Jul. 6, 2020, now abandoned,which is a continuation of U.S. application Ser. No. 16/217,636, filedon Dec. 12, 2018, now U.S. Pat. No. 10,736,718 B2, which is acontinuation of U.S. application Ser. No. 15/888,416, filed on Feb. 5,2018, now U.S. Pat. No. 10,383,711 B2, which is a continuation of U.S.application Ser. No. 14/764,087, filed on Jul. 28, 2015, now U.S. Pat.No. 9,962,244 B2, which is a U.S. National Stage of InternationalApplication No. PCT/EP2014/052842, filed on Feb. 13, 2014, which claimsthe benefit of U.S. Provisional Application No. 61/764,178, filed onFeb. 13, 2013, and the benefit of Danish Application No. PA 2013-70077,filed on Feb. 13, 2013. The entire contents of each of U.S. applicationSer. No. 18/183,358, U.S. application Ser. No. 16/921,414, U.S.application Ser. No. 16/217,636, U.S. application Ser. No. 15/888,416,U.S. application Ser. No. 14/764,087, International Application No.PCT/EP2014/052842, U.S. Provisional Application No. 61/764,178, andDanish Application No. PA 2013-70077 are hereby incorporated herein byreference in their entirety.

FIELD OF THE APPLICATION

The application relates to three dimensional (3D) scanning of thesurface geometry and surface color of objects. A particular applicationis within dentistry, particularly for intraoral scanning.

BACKGROUND

3D scanners are widely known from the art, and so are intraoral dental3D scanners (e.g., Sirona Cerec, Cadent Itero, 3Shape TRIOS).

The ability to record surface color is useful in many applications. Forexample, in dentistry, the user can differentiate types of tissue ordetect existing restorations. For example, in materials inspection, theuser can detect surface abnormalities such as crystallization defects ordiscoloring. None of the above is generally possible from surfacegeometry information alone.

WO2010145669 mentions the possibility of recording color. In particular,several sequential images, each taken for an illumination in a differentcolor—typically blue, green, and red—are combined to form a syntheticcolor image. This approach hence requires means to change light sourcecolor, such as color filters. Furthermore, in handheld use, the scannerwill move relative to the scanned object during the illuminationsequence, reducing the quality of the synthetic color image.

Also, U.S. Pat. Nos. 7,698,068 and 8,102,538 (Cadent Inc.) describe anintraoral scanner that records both geometry data and texture data withone or more image sensor(s). However, there is a slight delay betweenthe color and the geometry recording, respectively. U.S. Pat. No.7,698,068 requires sequential illumination in different colors to form asynthetic image, while U.S. Pat. No. 8,102,538 mentions white light as apossibility, however from a second illumination source or recorded by asecond image sensor, the first set being used for recording thegeometry.

WO2012083967 discloses a scanner for recording geometry data and texturedata with two separate cameras. While the first camera has a relativelyshallow depth of field as to provide focus scanning based on multipleimages, the second camera has a relatively large depth of field as toprovide color texture information from a single image.

Color-recording scanning confocal microscopes are also known from theprior art (e.g., Keyence VK9700; see also JP2004029373). A white lightillumination system along with a color image sensor is used forrecording 2D texture, while a laser beam forms a dot that is scanned,i.e., moved over the surface and recorded by a photomultiplier,providing the geometry data from many depth measurements, one for eachposition of the dot. The principle of a moving dot requires the measuredobject not to move relative to the microscope during measurement, andhence is not suitable for handheld use.

SUMMARY

One aspect of this application is to provide a scanner system and amethod for recording surface geometry and surface color of an object,and where surface geometry and surface color are derived from the samecaptured 2D images.

One aspect of this application is to provide a scanner system forrecording surface geometry and surface color of an object, and whereinall 2D images are captured using the same color image sensor.

One aspect of this application is to provide a scanner system and amethod for recording surface geometry and surface color of an object, inwhich the information relating to the surface geometry and to thesurface color are acquired simultaneously such that an alignment of datarelating to the recorded surface geometry and data relating to therecorded surface color is not required in order to generate a digital 3Drepresentation of the object expressing both color and geometry of theobject.

Disclosed is a scanner system for recording surface geometry and surfacecolor of an object, the scanner system comprising:

-   -   a multichromatic light source configured for providing a        multichromatic probe light for illumination of the object,    -   a color image sensor comprising an array of image sensor pixels        for capturing one or more 2D images of light received from said        object, and    -   a data processing system configured for deriving both surface        geometry information and surface color information for a block        of said image sensor pixels at least partly from one 2D image        recorded by said color image sensor.

Disclosed is a method of recording surface geometry and surface color ofan object, the method comprising:

-   -   obtaining a scanner system comprising a multichromatic light        source and a color image sensor comprising an array of image        sensor pixels;    -   illuminating the surface of said object with multichromatic        probe light from said multichromatic light source;    -   capturing a series of 2D images of said object using said color        image sensor; and    -   deriving both surface geometry information and surface color        information for a block of said image sensor pixels at least        partly from one captured 2D image.

In the context of the present application, the phrase “surface color”may refer to the apparent color of an object surface and thus in somecases, such as for semi-transparent or semi-translucent objects such asteeth, be caused by light from the object surface and/or the materialbelow the object surface, such as material immediately below the objectsurface.

In the context of the present application, the phrase “derived at leastpartly from one 2D image” refers to the situation where the surfacegeometry information for a given block of image sensor pixels at leastin part is derived from one 2D image and where the corresponding surfacecolor information at least in part is derived from the same 2D image.The phase also covers cases where the surface geometry information for agiven block of image sensor pixels at least in part is derived from aplurality of 2D images of a series of captured 2D images and where thecorresponding surface color information at least in part is derived fromthe same 2D images of that series of captured 2D images.

An advantage of deriving both surface geometry information and surfacecolor information for a block of said image sensor pixels at leastpartly from one 2D image is that a scanner system having only one imagesensor can be realized.

It is an advantage that the surface geometry information and the surfacecolor information are derived at least partly from one 2D image, sincethis inherently provides that the two types of information are acquiredsimultaneously. There is hence no requirement for an exact timing of theoperation of two color image sensors, which may the case when one imagesensor is used for the geometry recording and another for colorrecording. Equally there is no need for an elaborate calculationaccounting for significant differences in the timing of capturing of 2Dimages from which the surface geometry information is derived and thetiming of the capturing of 2D images from which the surface colorinformation is derived.

The present application discloses is a significant improvement over thestate of the art in that only a single image sensor and a singlemultichromatic light source is required, and that surface color andsurface geometry for at least a part of the object can be derived fromthe same 2D image or 2D images, which also means that alignment of colorand surface geometry is inherently perfect. In the scanner systemaccording to the present application, there is no need for taking intoaccount or compensating for relative motion of the object and scannersystem between obtaining surface geometry and surface color. Since thesurface geometry and the surface color are obtained at precisely thesame time, the scanner system automatically maintains its spatialdisposition with respect to the object surface while obtaining thesurface geometry and the surface color. This makes the scanner system ofthe present application suitable for handheld use, for example as anintraoral scanner, or for scanning moving objects.

In some embodiments, the data processing system is configured forderiving surface geometry information and surface color information forsaid block of image sensor pixels from a series of 2D images, such asfrom a plurality of the 2D images in a series of captured 2D images.I.e., the data processing system is capable of analyzing a plurality ofthe 2D images in a series of captured 2D images in order to derive thesurface geometry information for a block of image sensor pixels and toalso derive surface color information from at least one of the 2D imagesfrom which the surface geometry information is derived.

In some embodiments, the data processing system is configured forderiving surface color information from a plurality of 2D images of aseries of captured 2D images and for deriving surface geometryinformation from at least one of the 2D images from which the surfacecolor information is derived.

In some embodiments, the data processing system is configured forderiving surface geometry information from a plurality of 2D images of aseries of captured 2D images and for deriving surface color informationfrom at least one of the 2D images from which the surface geometryinformation is derived.

In some embodiments, the set of 2D images from which surface colorinformation is derived from is identical to the set of 2D images fromwhich surface geometry information is derived from.

In some embodiments, the data processing system is configured forgenerating a sub-scan of a part of the object surface based on surfacegeometry information and surface color information derived from aplurality of blocks of image sensor pixels. The sub-scan expresses atleast the geometry of the part of the object and typically one sub-scanis derived from one stack of captured 2D images.

In some embodiments, all 2D images of a captured series of images areanalyzed to derive the surface geometry information for each block ofimage sensor pixels on the color image sensor.

For a given block of image sensor pixels the corresponding portions ofthe captured 2D images in the stack may be analyzed to derive thesurface geometry information and surface color information for thatblock.

In some embodiments, the surface geometry information relates to wherethe object surface is located relative to the scanner system coordinatesystem for that particular block of image sensor pixels.

One advantage of the scanner system and the method of the currentapplication is that the information used for generating the sub-scanexpressing both geometry and color of the object (as seen from one view)are obtained concurrently.

Sub-scans can be generated for a number of different views of the objectsuch that they together cover the part of the surface.

In some embodiments, the data processing system is configured forcombining a number of sub-scans to generate a digital 3D representationof the object. The digital 3D representation of the object thenpreferably expresses both the recorded geometry and color of the object.

The digital 3D representation of the object can be in the form of a datafile. When the object is a patient's set of teeth the digital 3Drepresentation of this set of teeth can e.g., be used for CAD/CAMmanufacture of a physical model of the patient's set teeth.

The surface geometry and the surface color are both determined fromlight recorded by the color image sensor.

In some embodiments, the light received from the object originates fromthe multichromatic light source, i.e., it is probe light reflected orscattered from the surface of the object.

In some embodiments, the light received form the object comprisesfluorescence excited by the probe light from the multichromatic lightsource, i.e., fluorescence emitted by fluorescent materials in theobject surface.

In some embodiments, a second light source is used for the excitation offluorescence while the multichromatic light source provides the lightfor obtaining the geometry and color of the object.

The scanner system preferably comprises an optical system configured forguiding light emitted by the multichromatic light source towards theobject to be scanned and for guiding light received from the object tothe color image sensor such that the 2D images of said object can becaptured by said color image sensor.

In some embodiments, the scanner system comprises a first opticalsystem, such as an arrangement of lenses, for transmitting the probelight from the multichromatic light source towards an object and asecond optical system for imaging light received from the object at thecolor image sensor.

In some embodiments, single optical system images the probe light ontothe object and images the object, or at least a part of the object, ontothe color image sensor, preferably along the same optical axis, howeverin opposite directions along optical axis. The scanner may comprise atleast one beam splitter located in the optical path, where the beamsplitter is arranged such that it directs the probe light from themultichromatic light source towards the object while it directs lightreceived from the object towards the color image sensor.

Several scanning principles are suitable, such as triangulation andfocus scanning.

In some embodiments, the scanner system is a focus scanner systemoperating by translating a focus plane along an optical axis of thescanner system and capturing the 2D images at different focus planepositions such that each series of captured 2D images forms a stack of2D images. The focus plane position is preferably shifted along anoptical axis of the scanner system, such that 2D images captured at anumber of focus plane positions along the optical axis forms said stackof 2D images for a given view of the object, i.e., for a givenarrangement of the scanner system relative to the object. After changingthe arrangement of the scanner system relative to the object a new stackof 2D images for that view can be captured. The focus plane position maybe varied by means of at least one focus element, e.g., a moving focuslens.

In some focus scanner embodiments, the scanner system comprises apattern generating element configured for incorporating a spatialpattern in said probe light.

In some embodiments, the pattern generating element is configured toprovide that the probe light projected by scanner system onto the objectcomprises a pattern consisting of dark sections and sections with lighthaving a wavelength distribution according to the wavelengthdistribution of the multichromatic light source.

In some embodiments, the multichromatic light source comprises abroadband light source, such as a white light source.

In some embodiments, the pixels of the color image sensor and thepattern generating element are configured to provide that each pixelcorresponds to a single bright or dark region of the spatial patternincorporated in said probe light.

For a focus scanner system, the surface geometry information for a givenblock of image sensor pixels is derived by identifying at which distancefrom the scanner system the object surface is in focus for that block ofimage sensor pixels.

In some embodiments, deriving the surface geometry information andsurface color information comprises calculating for several 2D images,such as for several 2D images in a captured stack of 2D images, acorrelation measure between the portion of the 2D image captured by saidblock of image sensor pixels and a weight function. Here the weightfunction is preferably determined based on information of theconfiguration of the spatial pattern. The correlation measure may becalculated for each 2D image of the stack.

The scanner system may comprise means for evaluating a correlationmeasure at each focus plane position between at least one image pixeland a weight function, where the weight function is determined based oninformation of the configuration of the spatial pattern.

In some embodiments, deriving the surface geometry information and thesurface color information for a block of image sensor pixels comprisesidentifying the position along the optical axis at which thecorresponding correlation measure has a maximum value. The positionalong the optical axis at which the corresponding correlation measurehas a maximum value may coincide with the position where a 2D image hasbeen captured but it may even more likely be in between two neighboring2D images of the stack of 2D images.

Determining the surface geometry information may then relate tocalculating a correlation measure of the spatially structured lightsignal provided by the pattern with the variation of the pattern itself(which we term reference) for every location of the focus plane andfinding the location of an extremum of this stack of 2D images. In someembodiments, the pattern is static. Such a static pattern can forexample be realized as a chrome-on-glass pattern.

One way to define the correlation measure mathematically with a discreteset of measurements is as a dot product computed from a signal vector,I=(l1, . . . , ln), with n>1 elements representing sensor signals and areference vector, f=(f1, . . . , fn), of reference weights. Thecorrelation measure A is then given by

$A = {{f \cdot I} = {\sum\limits_{i = 1}^{n}{f_{i}I_{i}}}}$

The indices on the elements in the signal vector represent sensorsignals that are recorded at different pixels, typically in a block ofpixels. The reference vector f can be obtained in a calibration step.

By using knowledge of the optical system used in the scanner, it ispossible to transform the location of an extremum of the correlationmeasure, i.e., the focus plane into depth data information, on a pixelblock basis. All pixel blocks combined thus provide an array of depthdata. In other words, depth is along an optical path that is known fromthe optical design and/or found from calibration, and each block ofpixels on the image sensor represents the end point of an optical path.Therefore, depth along an optical path, for a bundle of paths, yields asurface geometry within the field of view of the scanner, i.e., asub-scan for the present view.

It can be advantageous to smooth and interpolate the series ofcorrelation measure values, such as to obtain a more robust and accuratedetermination of the location of the maximum.

In some embodiments, the generating a sub-scan comprises determining acorrelation measure function describing the variation of the correlationmeasure along the optical axis for each block of image sensor pixels andidentifying for the position along the optical axis at which thecorrelation measure functions have their maximum value for the block.

In some embodiments, the maximum correlation measure value is thehighest calculated correlation measure value for the block of imagesensor pixels and/or the highest maximum value of the correlationmeasure function for the block of image sensor pixels.

For example, a polynomial can be fitted to the values of A for a pixelblock over several images on both sides of the recorded maximum, and alocation of a deducted maximum can be found from the maximum of thefitted polynomial, which can be in between two images. The deductedmaximum is subsequently used as depth data information when deriving thesurface geometry from the present view, i.e., when deriving a sub-scanfor the view.

In some embodiments, the data processing system is configured fordetermining a color for a point on a generated sub-scan based on thesurface color information of the 2D image of the series in which thecorrelation measure has its maximum value for the corresponding block ofimage sensor pixels. The color may e.g., be read as the RGB values forpixels in said block of image sensor pixels.

In some embodiments, the data processing system is configured forderiving the color for a point on a generated sub-scan based on thesurface color information of the 2D images in the series in which thecorrelation measure has its maximum value for the corresponding block ofimage sensor pixels and on at least one additional 2D image, such as aneighboring 2D image from the series of captured 2D images. The surfacecolor information is still derived from at least one of the 2D imagesfrom which the surface geometry information is derived.

In some embodiments, the data processing system is configured forinterpolating surface color information of at least two 2D images in aseries when determining the sub-scan color, such as an interpolation ofsurface color information of neighboring 2D images in a series.

In some embodiments, the data processing system is configured forcomputing a smoothed color for a number of points of the sub-scan, wherethe computing comprises an averaging of sub-scan colors of differentpoints, such as a weighted averaging of the colors of the surroundingpoints on the sub-scan.

Surface color information for a block of image sensor pixels is at leastpartially derived from the same image from which surface geometryinformation is derived. In case the location of the maximum of A isrepresented by a 2D image, then also color is derived from that sameimage. In case the location of the maximum of A is found byinterpolation to be between two images, then at least one of those twoimages should be used to derive color, or both images usinginterpolation for color also. It is also possible to average color datafrom more than two images used in the determination of the location ofthe maximum of the correlation measure, or to average color from asubset or superset of multiple images used to derive surface geometry.In any case, some image sensor pixels readings are used to derive bothsurface color and surface geometry for at least a part of the scannedobject.

Typically, there are three color filters, so the overall color iscomposed of three contributions, such as red, green, and blue, or cyan,magenta, and yellow. Note that color filters typically allow a range ofwavelengths to pass, and there is typically cross-talk between filters,such that, for example, some green light will contribute to theintensity measured in pixels with red filters.

For an image sensor with a color filter array, a color component cwithin a pixel block can be obtained as

$c_{j} = {\sum\limits_{i = 1}^{n}{g_{j,i}I_{i}}}$

where g_(j,i)=1 if pixel i has a filter for color c_(j), 0 otherwise.For an RGB filter array like in a Bayer pattern, j is one of red, green,or blue. Further weighting of the individual color components, i.e.,color calibration, may be required to obtain natural color data,typically as compensation for varying filter efficiency, illuminationsource efficiency, and different fraction of color components in thefilter pattern. The calibration may also depend on focus plane locationand/or position within the field of view, as the mixing of the lightsource component colors may vary with those factors.

In some embodiments, surface color information is obtained for everypixel in a pixel block. In color image sensors with a color filter arrayor with other means to separate colors such as diffractive means,depending on the color measured with a particular pixel, an intensityvalue for that color is obtained. In other words, in this case aparticular pixel has a color value only for one color. Recentlydeveloped color image sensors allow measurement of several colors in thesame pixel, at different depths in the substrate, so in that case, aparticular pixel can yield intensity values for several colors. Insummary, it is possible to obtain a resolution of the surface color datathat is inherently higher than that of the surface geometry information.

In the embodiments where the resolution of the derived color is higherthan the resolution of the surface geometry for the generated digital 3Drepresentation of the object, a pattern will be visible when at leastapproximately in focus, which preferably is the case when color isderived. The image can be filtered such as to visually remove thepattern, however at a loss of resolution. In fact, it can beadvantageous to be able to see the pattern for the user. For example, inintraoral scanning, it may be important to detect the position of amargin line, the rim or edge of a preparation. The image of the patternoverlaid on the geometry of this edge is sharper on a side that is seenapproximately perpendicular, and more blurred on the side that is seenat an acute angle. Thus, a user, who in this example typically is adentist or dental technician, can use the difference in sharpness tomore precisely locate the position of the margin line than may bepossible from examining the surface geometry alone.

High spatial contrast of an in-focus pattern image on the object isdesirable to obtain a good signal to noise ratio of the correlationmeasure on the color image sensor. Improved spatial contrast can beachieved by preferential imaging of the specular surface reflection fromthe object on the color image sensor. Thus, some embodiments comprisemeans for preferential/selective imaging of specularly reflected light.This may be provided if the scanner further comprises means forpolarizing the probe light, for example by means of at least onepolarizing beam splitter.

In some embodiments, the polarizing optics is coated such as to optimizepreservation of the circular polarization of a part of the spectrum ofthe multichromatic light source that is used for recording the surfacegeometry.

The scanner system may further comprise means for changing thepolarization state of the probe light and/or the light received from theobject. This can be provided by means of a retardation plate, preferablylocated in the optical path. In some embodiments, the retardation plateis a quarter wave retardation plate.

Especially for intraoral applications where the scanned object e.g., isthe patient's set or teeth, the scanner can have an elongated tip, withmeans for directing the probe light and/or imaging an object. This maybe provided by means of at least one folding element. The foldingelement could be a light reflecting element such as a mirror or a prism.The probe light then emerges from the scanner system along an opticalaxis at least partly defined by the folding element.

For a more in-depth description of the focus scanning technology, seeWO2010145669.

In some embodiments, the data processing system is configured fordetermining the color of a least one point of the generated digital 3Drepresentation of the object, such that the digital 3D representationexpresses both geometry and color profile of the object. Color may bedetermined for several points of the generated digital 3D representationsuch that the color profile of the scanned part of the object isexpressed by the digital 3D representation.

In some embodiments determining the object color comprises computing aweighted average of color values derived for corresponding points inoverlapping sub-scans at that point of the object surface. This weightedaverage can then be used as the color of the point in the digital 3Drepresentation of the object.

In some embodiments the data processing system is configured fordetecting saturated pixels in the captured 2D images and for mitigatingor removing the error in the derived surface color information or thesub-scan color caused by the pixel saturation.

In some embodiments the error caused by the saturated pixel is mitigatedor removed by assigning a low weight to the surface color information ofthe saturated pixel in the computing of the smoothed color of a sub-scanand/or by assigning a low weight to the color of a sub-scan computedbased on the saturated pixel.

In some embodiments, the data processing system is configured forcomparing the derived surface color information of sections of thecaptured 2D images and/or of the generated sub-scans of the object withpredetermined color ranges for teeth and for oral tissue, and forsuppressing the red component of the derived surface color informationor sub-scan color for sections where the color is not in one of the twopredetermined color ranges.

The scanner system disclosed here comprises a multichromatic lightsource, for example a white light source, for example a multi-die LED.

Light received from the scanned object, such as probe light returnedfrom the object surface or fluorescence generated by the probe light byexciting fluorescent parts of the object, is recorded by the color imagesensor. In some embodiments, the color image sensor comprises a colorfilter array such that every pixel in the color image sensor is acolor-specific filter. The color filters are preferably arranged in aregular pattern, for example where the color filters are arrangedaccording to a Bayer color filter pattern. The image data thus obtainedare used to derive both surface geometry and surface color for eachblock of pixels. For a focus scanner utilizing a correlation measure,the surface geometry may be found from an extremum of the correlationmeasure as described above. In some embodiments, the surface geometry isderived from light in a first part of the spectrum of the probe lightprovided by the multichromatic light source.

Preferably, the color filters are aligned with the image sensor pixels,preferably such that each pixel has a color filter for a particularcolor only.

In some embodiments, the color filter array is such that its proportionof pixels with color filters that match the first part of the spectrumis larger than 50%.

In some embodiments, the surface geometry information is derived fromlight in a selected wavelength range of the spectrum provided by themultichromatic light source. The light in the other wavelength ranges ishence not used to derive the surface geometry information. This providesthe advantage that chromatic dispersion of optical elements in theoptical system of the scanner system does not influence the scanning ofthe object.

It can be preferable to compute the surface geometry only from pixelswith one or two types of color filters. A single color requires noachromatic optics and is thus provides for a scanner that is easier andcheaper to build. Furthermore, folding elements can generally notpreserve the polarization state for all colors equally well. When onlysome color(s) is/are used to compute surface geometry, the referencevector f will contain zeros for the pixels with filters for the othercolor(s). Accordingly, the total signal strength is generally reduced,but for large enough blocks of pixels, it is generally still sufficient.Preferentially, the pixel color filters are adapted for littlecross-talk from one color to the other(s). Note that even in theembodiments computing geometry from only a subset of pixels, color ispreferably still computed from all pixels.

In some embodiments, the color image sensor comprises a color filterarray comprising at least three types of colors filters, each allowinglight in a known wavelength range, W1, W2, and W3 respectively, topropagate through the color filter.

In some embodiments, the color filter array is such that its proportionof pixels with color filters that match the selected wavelength range ofthe spectrum is larger than 50%, such a wherein the proportion equals32/36, 60/64 or 96/100.

In some embodiments, the selected wavelength range matches the W2wavelength range.

In some embodiments, the color filter array comprises a plurality ofcells of 6×6 color filters, where the color filters in positions (2,2)and (5,5) of each cell are of the W1 type, the color filters inpositions (2,5) and (5,2) are of the W3 type. Here a W1 type of filteris a color tilter that allows light in the known wavelength range W1 topropagate through the color filter, and similar for W2 and W3 type offilters. In some embodiments, the remaining 32 color filters in the 6×6cell are of the W2 type.

In a RGB color system, W1 may correspond to red light, W2 to greenlight, and W3 to blue light.

In some embodiments, the scanner is configured to derive the surfacecolor with a higher resolution than the surface geometry.

In some embodiments, the higher surface color resolution is achieved bydemosaicing, where color values for pixel blocks may be demosaiced toachieve an apparently higher resolution of the color image than ispresent in the surface geometry. The demosaicing may operate on pixelblocks or individual pixels.

In case a multi-die LED or another illumination source comprisingphysically or optically separated light emitters is used, it ispreferable to aim at a Köhler type illumination in the scanner, i.e.,the illumination source is defocused at the object plane in order toachieve uniform illumination and good color mixing for the entire fieldof view. In case color mixing is not perfect and varies with focal planelocation, color calibration of the scanner will be advantageous.

In some embodiments, the pattern generating element is configured toprovide that the spatial pattern comprises alternating dark and brightregions arranged in a checkerboard pattern. The probe light provided bythe scanner system then comprises a pattern consisting of dark sectionsand sections with light having the same wavelength distribution as themultichromatic light source.

In order to obtain a digital 3D representation expressing both surfacegeometry and color representation of an object, i.e., a colored digital3D representation of said part of the object surface, typically severalsub-scans, i.e., partial representations of the object, have to becombined, where each sub-scans presents one view of the object. Asub-scan expressing a view from a given relative position preferablyrecords the geometry and color of the object surface as seen from thatrelative position.

For a focus scanner, a view corresponds to one pass of the focusingelement(s), i.e., for a focus scanner each sub-scan is the surfacegeometry and color derived from the stack of 2D images recorded duringthe pass of the focus plane position between its extremum positions.

The surface geometry found for various views can be combined byalgorithms for stitching and registration as widely known in theliterature, or from known view positions and orientations, for examplewhen the scanner is mounted on axes with encoders. Color can beinterpolated and averaged by methods such as texture weaving, or bysimply averaging corresponding color components in multiple views of thesame location on the surface. Here, it can be advantageous to accountfor differences in apparent color due to different angles of incidenceand reflection, which is possible because the surface geometry is alsoknown. Texture weaving is described by e.g., Callieri M, Cignoni P,Scopigno R. “Reconstructing textured meshes from multiple range rgbmaps.” VMV 2002, Erlangen, Nov. 20-22, 2002.

In some embodiments, the scanner and/or the scanner system is configuredfor generating a sub-scan of the object surface based on the obtainedsurface color and surface geometry. In some embodiments, the scannerand/or the scanner system is configured for combining sub-scans of theobject surface obtained from different relative positions to generate adigital 3D representation expressing the surface geometry and color ofat least part of the object.

In some embodiments, the combination of sub-scans of the object toobtain the digital 3D representation expressing surface geometry andcolor comprises computing the color in each surface point as a weightedaverage of corresponding points in all overlapping sub-scans at thatsurface point. The weight of each sub-scan in the sum may be determinedby several factors, such as the presence of saturated pixel values orthe orientation of the object surface with respect to the scanner whenthe sub-scan is recorded.

Such a weighted average is advantageous in cases where some scannerpositions and orientations relative to the object will give a betterestimate of the actual color than other positions and orientations. Ifthe illumination of the object surface is uneven this can to some degreealso be compensated for by weighting the best illuminated parts higher.

In some embodiments, the data processing system of the scanner systemcomprises an image processor configured for performing a post-processingof the surface geometry, the surface color readings, or the derivedsub-scan or the digital 3D representation of the object. The scannersystem may be configured for performing the combination of the sub-scansusing e.g., computer implemented algorithms executed by the imageprocessor.

The scanner system may be configured for performing the combination ofthe sub-scans using e.g. computer implemented algorithms executed by thedata processing system as part of the post-processing of the surfacegeometry, surface color, sub-scan and/or the digital 3D representation,i.e. the post-processing comprises computing the color in each surfacepoint as a weighted average of corresponding points in all overlappingsub-scans at that surface point.

Saturated pixel values should preferably have a low weight to reduce theeffect of highlights on the recording of the surface color. The colorfor a given part of the surface should preferably be determinedprimarily from 2D images where the color can be determined preciselywhich is not the case when the pixel values are saturated.

In some embodiments, the scanner and/or scanner system is configured fordetecting saturated pixels in the captured 2D images and for mitigatingor removing the error in the obtained color caused by the pixelsaturation. The error caused by the saturated pixel may be mitigated orremoved by assigning a low weight to the saturated pixel in the weightedaverage.

Specularly reflected light has the color of the light source rather thanthe color of the object surface. If the object surface is not a purewhite reflector, then specular reflections can hence be identified asthe areas where the pixel color closely matches the light source color.When obtaining the surface color, it is therefore advantageous to assigna low weight to pixels or pixel groups whose color values closely matchthe color of the multichromatic light source in order to compensate forsuch specular reflections.

Specular reflections may also be a problem when intra orally scanning apatient's set of teeth since teeth rarely are completely white. It mayhence be advantageous to assume that for pixels where the readings fromthe color images sensor indicate that the surface of the object is apure white reflector, the light recorded by this pixel group is causedby a specular reflection from the teeth or the soft tissue in the oralcavity and accordingly assign a low weight to these pixels to compensatefor the specular reflections.

In some embodiments, the compensation for specular reflections from theobject surface is based on information derived from a calibration of thescanner in which a calibration object e.g., in the form of a pure whitereflector is scanned. The color image sensor readings then depend on thespectrum of the multichromatic light source and on the wavelengthdependence of the scanner's optical system caused by e.g., a wavelengthdependent reflectance of mirrors in the optical system. If the opticalsystem guides light equally well for all wavelengths of themultichromatic light source, the color image sensor will record thecolor (also referred to as the spectrum) of the multichromatic lightsource when the pure white reflector is scanned.

In some embodiments, compensating for the specular reflections from thesurface is based on information derived from a calculation based on thewavelength dependence of the scanner's optical system, the spectrum ofthe multichromatic light source and a wavelength dependent sensitivityof the color image sensor. In some embodiments, the scanner comprisesmeans for optically suppressing specularly reflected light to achievebetter color measurement. This may be provided if the scanner furthercomprises means for polarizing the probe light, for example by means ofat least one polarizing beam splitter. When scanning inside an oralcavity there may be red ambient light caused by probe light illuminationof surrounding tissue, such as the gingiva, palette, tongue, or buccaltissue. In some embodiments, the scanner and/or scanner system is henceconfigured for suppressing the red component in the recorded 2D images.

In some embodiments, the scanner and/or scanner system is configured forcomparing the color of sections of the captured 2D images and/or of thesub-scans of the object with predetermined color ranges for teeth andfor oral tissue, respectively, and for suppressing the red component ofthe recorded color for sections where the color is not in either one ofthe two predetermined color ranges. The teeth may e.g., be assumed to beprimarily white with one ratio between the intensity of the differentcomponents of the recorded image, e.g., with one ratio between theintensity of the red component and the intensity of the blue and/orgreen components in a RGB configuration, while oral tissue is primarilyreddish with another ratio between the intensity of the components. Whena color recorded for a region of the oral cavity shows a ratio whichdiffers from both the predetermined ratio for teeth and thepredetermined ratio for tissue, this region is identified as a toothregion illuminated by red ambient light and the red component of therecorded image is suppressed relative to the other components, either byreducing the recorded intensity of the red signal or by increasing therecorded intensities of the other components in the image.

In some embodiments, the color of points with a surface normal directlytowards the scanner are weighted higher than the color of points wherethe surface normal is not directed towards the scanner. This has theadvantage that points with a surface normal directly towards the scannerwill to a higher degree be illuminated by the white light from thescanner and not by the ambient light.

In some embodiments, the color of points with a surface normal directlytowards the scanner are weighted lower if associated with specularreflections.

In some embodiments the scanner is configured for simultaneouslycompensating for different effects, such as compensating for saturatedpixels and/or for specular reflections and/or for orientation of thesurface normal. This may be done by generally raising the weight for aselection of pixels or pixel groups of a 2D image and by reducing theweight for a fraction of the pixels or pixel groups of said selection.

In some embodiments, the method comprises a processing of recorded 2Dimages, a sub-scan, or the generated 3D representations of the part ofthe object, where said processing comprises

-   -   compensating for pixel saturation by omitting or reducing the        weight of saturated pixels when deriving the surface color,        and/or    -   compensating for specular reflections when deriving the surface        color by omitting or reducing the weight of pixels whose color        values closely matches the light source color, and/or    -   compensating for red ambient light by comparing surface color        information of the 2D images with predetermined color ranges,        and suppressing the red component of the recorded color if this        is not within a predetermined color range.

Disclosed is a method of using the disclosed scanner system to displaycolor texture on the generated digital 3D representation of the object.It is advantageous to display the color data as a texture on the digital3D representation, for example on a computer screen. The combination ofcolor and geometry is a more powerful conveyor of information thaneither type of data alone. For example, dentists can more easilydifferentiate between different types of tissue. In the rendering of thesurface geometry, appropriate shading can help convey the surfacegeometry on the texture, for example with artificial shadows revealingsharp edges better than texture alone could do.

When the multichromatic light source is a multi-die LED or similar, thescanner system can also be used to detect fluorescence. Disclosed is amethod of using the disclosed scanner system to display fluorescence onsurface geometry.

In some embodiments, the scanner is configured for exciting fluorescenceon said object by illuminating it with only a subset of the LED dies inthe multi-die LED, and where said fluorescence is recorded by only orpreferentially reading out only those pixels in the color image sensorthat have color filters at least approximately matching the color of thefluoresced light, i.e. measuring intensity only in pixels of the imagesensors that have filters for longer-wavelength light. In other words,the scanner is capable of selectively activating only a subset of theLED dies in the multi-die LED and of only recording or preferentiallyreading out only those pixels in the color image sensor that have colorfilters at a higher wavelength than that of the subset of the LED dies,such that light emitted from the subset of LED dies can excitefluorescent materials in the object and the scanner can record thefluorescence emitted from these fluorescent materials. The subset of thedies preferably comprises one or more LED dies which emits light withinthe excitation spectrum of the fluorescent materials in the object, suchas an ultraviolet, a blue, a green, a yellow or a red LED die. Suchfluorescence measurement yields a 2D data array much like the 2D colorimage, however unlike the 2D image it cannot be taken concurrently withthe surface geometry. For a slow-moving scanner, and/or with appropriateinterpolation, the fluorescence image can still be overlaid the surfacegeometry. It is advantageous to display fluorescence on teeth because itcan help detect caries and plaque.

In some embodiments, the data processing system comprises amicroprocessor unit configured for extracting the surface geometryinformation from 2D images obtained by the color image sensor and fordetermining the surface color from the same images.

The data processing system may comprise units distributed in differentparts of the scanner system. For a scanner system comprising a handheldpart connected to a stationary unit, the data processing system may forexample comprise one unit integrated in the handheld part and anotherunit integrated in the stationary unit. This can be advantageous when adata connection for transferring data from the handheld unit to thestationary unit has a bandwidth which cannot handle the data stream fromthe color image sensor. A preliminary data processing in the handheldunit can then reduce the amount of data which must be transferred viathe data connection.

In some embodiments, the data processing system comprises a computerreadable medium on which is stored computer implemented algorithms forperforming said post-processing.

In some embodiments, a part of the data processing system is integratedin a cart or a personal computer.

Disclosed is a method of using the disclosed scanner system to averagecolor and/or surface geometry from several views, where each viewrepresents a substantially fixed relative orientation of scanner andobject.

Disclosed is a method using the disclosed scanner system to combinecolor and/or surface geometry from several views, where each viewrepresents a substantially fixed relative orientation of scanner andobject, such as to achieve a more complete coverage of the object thanwould be possible in a single view.

Disclosed is a scanner for obtaining surface geometry and surface colorof an object, the scanner comprising:

-   -   a multichromatic light source configured for providing a probe        light, and    -   a color image sensor comprising an array of image sensor pixels        for recording one or more 2D images of light received from said        object,        where at least for a block of said image sensor pixels, both        surface color and surface geometry of a part of the object are        derived at least partly from one 2D image recorded by said color        image sensor

Disclosed is a scanner system for recording surface geometry and surfacecolor of an object, the scanner system comprising:

-   -   a multichromatic light source configured for providing a        multichromatic probe light, and    -   a color image sensor comprising an array of image sensor pixels        for capturing one or more 2D images of light received from said        object,        where at least for a block of said image sensor pixels, both        surface color information and surface geometry information of a        part of the object are derived at least partly from one 2D image        captured by said color image sensor.

Disclosed is a scanner system for recording surface geometry and surfacecolor of an object, the scanner system comprising:

-   -   a multichromatic light source configured for providing a probe        light,    -   a color image sensor comprising an array of image sensor pixels,        and    -   an optical system configured for guiding light received from the        object to the color image sensor such that 2D images of said        object can be captured by said color image sensor;        wherein the scanner system is configured for capturing a number        of said 2D images of a part of the object and for deriving both        surface color information and surface geometry information of        the part of the object from at least one of said captured 2D        images at least for a block of said color image sensor pixels,        such that the surface color information and the surface geometry        information are obtained concurrently by the scanner.

Disclosed is a scanner system for recording surface geometry and surfacecolor of an object, the scanner system comprising:

-   -   a multichromatic light source configured for providing a probe        light;    -   a color image sensor comprising an array of image sensor pixels,        where the image sensor is arranged to capture 2D images of light        received from the object; and    -   an image processor configured for deriving both surface color        information and surface geometry information of at least a part        of the object from at least one of said 2D images captured by        the color image sensor.

Disclosed is a scanner system for recording surface geometry and surfacecolor of an object, said scanner system comprising

-   -   a scanner system according to any of the embodiments, where the        scanner system is configured for deriving surface color and        surface geometry of the object, and optionally for generating a        sub-scan or a digital 3D representation of the part of the        object; and    -   a data processing unit configured for post-processing surface        geometry and/or surface color readings from the color image        sensor, or for post-processing the generated sub-scan or digital        3D representation.

Disclosed is a method of recording surface geometry and surface color ofan object, the method comprising:

-   -   providing a scanner or scanner system according to any of the        embodiments;    -   illuminating the surface of said object with probe light from        said multichromatic light source;    -   recording one or more 2D images of said object using said color        image sensor; and    -   deriving both surface color and surface geometry of a part of        the object from at least some of said recorded 2D images at        least for a block of said image sensor pixels, such that the        surface color and surface geometry are obtained concurrently by        the scanner.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a handheld embodiment of a scanner system.

FIGS. 2A-2B show prior art pattern generating means and associatedreference weights.

FIGS. 3A-3B show a pattern generating means and associated referenceweights.

FIG. 4 shows a color filter array.

FIG. 5 shows a flow chart of a method.

FIGS. 6A-6C illustrate illustrates how surface geometry information andsurface geometry information can be derived.

FIG. 1 shows a handheld part of a scanner system with components insidea housing 100. The scanner comprises a tip which can be entered into acavity, a multichromatic light source in the form of a multi-die LED101, pattern generating element 130 for incorporating a spatial patternin the probe light, a beam splitter 140, color image sensor 180including an image sensor 181, electronics and potentially otherelements, an optical system typically comprising at least one lens, andthe image sensor. The light from the light source 101 travels back andforth through the optical system 150. During this passage, the opticalsystem images the pattern 130 onto the object being scanned 200 whichhere is a patient's set of teeth, and further images the object beingscanned onto the image sensor 181.

The image sensor 181 has a color filter array 1000. Although drawn as aseparate entity, the color filter array is typically integrated with theimage sensor, with a single-color filter for every pixel.

The lens system includes a focusing element 151 which can be adjusted toshift the focal imaging plane of the pattern on the probed object 200.In the example embodiment, a single lens element is shifted physicallyback and forth along the optical axis.

As a whole, the optical system provides an imaging of the pattern ontothe object being probed and from the object being probed to the camera.

The device may include polarization optics 160. Polarization optics canbe used to selectively image specular reflections and block outundesired diffuse signal from sub-surface scattering inside the scannedobject. The beam splitter 140 may also have polarization filteringproperties. It can be advantageous for optical elements to beanti-reflection coated.

The device may include folding optics, a mirror 170, which directs thelight out of the device in a direction different to the optical path ofthe lens system, e.g., in a direction perpendicular to the optical pathof the lens system.

There may be additional optical elements in the scanner, for example oneor more condenser lens in front of the light source 101.

In the example embodiment, the LED 101 is a multi-die LED with twogreen, one red, and one blue die. Only the green portion of the light isused for obtaining the surface geometry. Accordingly, the mirror 170 iscoated such as to optimize preservation of the circular polarization ofthe green light, and not that of the other colors. Note that duringscanning all dies within the LED are active, i.e., emitting light, sothe scanner emits apparently white light onto the scanned object 200.The LED may emit light at the different colors with differentintensities such that e.g., one color is more intense than the othercolors. This may be desired in order to reduce cross-talk between thereadings of the different color signals in the color image sensor. Incase that the intensity of e.g., the red and blue diodes in a RGB systemis reduced, the apparently white light emitted by the light source willappear greenish-white.

The scanner system further comprises a data processing system configuredfor deriving both surface geometry information and surface colorinformation for a block of pixels of the color image sensor 180 at leastpartly from one 2D image recorded by said color image sensor 180. Atleast part of the data processing system may be arranged in theillustrated handheld part of the scanner system. A part may also bearranged in an additional part of the scanner system, such as a cartconnected to the handheld part.

FIGS. 2A-2B show a section of a prior art pattern generating element 130that is applied as a static pattern in a spatial correlation embodimentof WO2010145669, as imaged on a monochromatic image sensor 180. Thepattern can be a chrome-on-glass pattern. The section shows only aportion of the pattern is shown, namely one period. This period isrepresented by a pixel block of 6 by 6 image pixels, and 2 by 2 patternfields. The fields drawn in gray in FIG. 2A are in actuality blackbecause the pattern mask is opaque for these fields; gray was onlychosen for visibility and thus clarity of the Figure. FIG. 2Billustrates the reference weights f for computing the spatialcorrelation measure A for the pixel block, where n=6×6=36, such that

$A = {\sum\limits_{i = 1}^{n}{f_{i}I_{i}}}$

where I are the intensity values measured in the 36 pixels in the pixelblock for a given image. Note that perfect alignment between imagesensor pixels and pattern fields is not required, but gives the bestsignal for the surface geometry measurement.

FIGS. 3A-3B show the extension of the principle in FIGS. 2A-2B to colorscanning. The pattern is the same as in FIGS. 2A-2B and so is the imagesensor geometry. However, the image sensor is a color image sensor witha Bayer color filter array. In FIG. 3A, pixels marked “B” have a bluecolor filter, while “G” indicates green and “R” red pixel filters,respectively. FIG. 3B shows the corresponding reference weights f. Notethat only green pixels have a non-zero value. This is so because onlythe green fraction of the spectrum is used for recording the surfacegeometry information.

For the pattern/color filter combination of FIGS. 3A-3B, a colorcomponent cc within a pixel block can be obtained as

$c_{j} = {\sum\limits_{i = 1}^{n}{g_{j,i}I_{i}}}$

where g_(j,i)=1 if pixel i has a filter for color c_(j), 0 otherwise.For an RGB color filter array like in the Bayer pattern, j is one ofred, green, or blue. Further weighting of the individual colorcomponents, i.e., color calibration, may be required to obtain naturalcolor data, typically as compensation for varying filter efficiency,illumination source efficiency, and different fraction of colorcomponents in the filter pattern. The calibration may also depend onfocus plane location and/or position within the field of view, as themixing of the LED's component colors may vary with those factors.

FIG. 4 shows an inventive color filter array with a higher fraction ofgreen pixels than in the Bayer pattern. In some embodiments, the colorfilter array comprises a plurality of cells of 6×6 color filters, withblue color filters in positions (2,2) and (5,5) of each cell, red colorfilters in positions (2,5) and (5,2), a and green color filters in allremaining positions of the cell.

Assuming that only the green portion of the illumination is used toobtain the surface geometry information, the filter of FIG. 4 willpotentially provide a better quality of the obtained surface geometrythan a Bayer pattern filter, at the expense of poorer colorrepresentation. The poorer color representation will however in manycases still be sufficient while the improved quality of the obtainedsurface geometry often is very advantageous.

FIG. 5 illustrates a flow chart 541 of a method of recording surfacegeometry and surface color of an object.

In step 542 a scanner system according to any of the previous claims isobtained.

In step 543 the object is illuminated with multichromatic probe light.In a focus scanning system utilizing a correlation measure orcorrelation measure function, a checkerboard pattern may be imposed onthe probe light such that information relating to the pattern can beused for determining surface geometry information from captured 2Dimages.

In step 544 a series of 2D images of said object is captured using saidcolor image sensor. The 2D images can be processed immediately or storedfor later processing in a memory unit.

In step 545 both surface geometry information and surface colorinformation are derived for a block of image sensor pixels at leastpartly from one captured 2D image. The information can e.g. be derivedusing the correlation measure approach as descried herein. The derivedinformation are combined to generate a sub-scan of the object in step546, where the sub-scan comprises, data expressing the geometry andcolor of the object as seen from one view.

In step 547 a digital 3D representation expressing both color andgeometry of the object is generated by combining several sub-scans. Thismay be done using known algorithms for sub-scan alignment such asalgorithms for stitching and registration as widely known in theliterature.

FIGS. 6A-6C illustrate how surface geometry information and surfacegeometry information can be derived at least from one 2D image for ablock of image sensor pixels.

The correlation measure is determined for all active image sensor pixelgroups on the color image sensor for every focus plane position, i.e.,for every 2D image of the stack. Starting by analyzing the 2D imagesfrom one end of the stack, the correlation measures for all active imagesensor pixel groups is determined and the calculated values are stored.

Progressing through the stack the correlation measures for each pixelgroup are determined and stored together with the previously storedvalues, i.e., the values for the previously analyzed 2D images.

A correlation measure function describing the variation of thecorrelation measure along the optical axis is then determined for eachpixel group by smoothing and interpolating the determined correlationmeasure values. For example, a polynomial can be fitted to the values offor a pixel block over several images on both sides of the recordedmaximum, and a location of a deducted maximum can be found from themaximum of the fitted polynomial, which can be in between two images.

The surface color information for the pixel group is derived from one ormore of the 2D images from which the position of the correlation measuremaximum was determined i.e., surface geometry information and surfacecolor information from a group of pixels of the color image sensor arederived from the same 2D images of the stack.

The surface color information can be derived from one 2D image. Themaximum value of the correlation measure for each group of pixels ismonitored along the analysis of the 2D images such that when a 2D imagehas been analyzed the values for the correlation measure for thedifferent pixels groups can be compared with the currently highest valuefor the previously analyzed 2D images. If the correlation measure is anew maximum value for that pixel group at least the portion of the 2Dimage corresponding to this pixel group is saved. Next time a highercorrelation value is found for that pixel group the portion of this 2Dimage is saved overwriting the previously stored image/sub-image.Thereby when all 2D images of the stack have been analyzed, the surfacegeometry information of the 2D images is translated into a series ofcorrelation measure values for each pixel group where a maximum value isrecorded for each block of image sensor pixels.

FIG. 6A illustrated a portion 661 of a stack of 2D images acquired usinga focus scanning system, where each 2D image is acquired at a differentfocal plane position. In each 2D image 662 a portion 663 correspondingto a block of image sensor pixels are indicated. The block correspondingto a set of coordinates (x_(i),y_(i)). The focus scanning system isconfigured for determining a correlation measure for each block of imagesensor pixels and for each 2D image in the stack. In FIG. 6B isillustrated the determined correlation measures 664 (here indicated byan “x”) for the block 663. Based on the determined correlation measures664 a correlation measure function 665 is calculated, here as apolynomial, and a maximum value for the correlation measure function isfound a position z_(i). The z-value for which the fitted polynomial hasa maximum (z_(i)) is identified as a point of the object surface. Thesurface geometry information derived for this block can then bepresented in the form of the coordinates (x_(i),y_(i),z_(i)), and bycombining the surface geometry information for several block of theimages sensor, then a sub-scan expressing the geometry of part of theobject can be created.

In FIG. 6C is illustrated a procedure for deriving the surface colorgeometry from two 2D images for each block of image sensor pixels. Two2D images are stored using the procedure described above and their RGBvalues for the pixel block are determined. In FIG. 6C the R-values 666are displayed. An averaged R-value 667 (as well as averaged G- andB-values) at the z_(i) position can then be determined by interpolationand used as surface color information for this block. This surface colorinformation is evidently derived from the same 2D image that thegeometry information at least in part was derived from.

EMBODIMENTS

-   -   1. A scanner system for recording surface geometry and surface        color of an object, the scanner system comprising:        -   a multichromatic light source configured for providing a            multichromatic probe light for illumination of the object,        -   a color image sensor comprising an array of image sensor            pixels for capturing one or more 2D images of light received            from said object, and        -   a data processing system configured for deriving both            surface geometry information and surface color information            for a block of said image sensor pixels at least partly from            one 2D image recorded by said color image sensor.    -   2. The scanner system according to embodiment 1, wherein the        data processing system is configured for deriving surface        geometry information and surface color information for said        block of image sensor pixels from a series of 2D images.    -   3. The scanner system according to embodiment 1 or 2, wherein        the data processing system is configured for generating a        sub-scan of a part of the object surface based on surface        geometry information and surface color information derived from        a plurality of blocks of image sensor pixels.    -   4. The scanner system according to any of embodiments 1 to 3,        wherein the data processing system is configured for combining a        number of sub-scans to generate a digital 3D representation of        the object.    -   5. The scanner system according to any of embodiments 2 to 5,        where the scanner system is a focus scanner system operating by        translating a focus plane along an optical axis of the scanner        system and capturing the 2D images at different focus plane        positions such that each series of captured 2D images forms a        stack of 2D images.    -   6. The scanner system according to any of the preceding        embodiments, where the scanner system comprises a pattern        generating element configured for incorporating a spatial        pattern in said probe light.    -   7. The scanner system according to any of the preceding        embodiments, where deriving the surface geometry information and        surface color information comprises calculating for several 2D        images a correlation measure between the portion of the 2D image        captured by said block of image sensor pixels and a weight        function, where the weight function is determined based on        information of the configuration of the spatial pattern.    -   8. The scanner system according to the preceding embodiment,        wherein deriving the surface geometry information and the        surface color information for a block of image sensor pixels        comprises identifying the position along the optical axis at        which the corresponding correlation measure has a maximum value.    -   9. The scanner system according to embodiment 7 or 8, wherein        generating a sub-scan comprises determining a correlation        measure function describing the variation of the correlation        measure along the optical axis for each block of image sensor        pixels and identifying for the position along the optical axis        at which the correlation measure functions have their maximum        value for the block.    -   10. The scanner system according to the preceding embodiment,        where the maximum correlation measure value is the highest        calculated correlation measure value for the block of image        sensor pixels and/or the highest maximum value of the        correlation measure function for the block of image sensor        pixels.    -   11. The scanner system according to any of the preceding        embodiments, wherein the data processing system is configured        for determining a sub-scan color for a point on a generated        sub-scan based on the surface color information of the 2D image        in the series in which the correlation measure has its maximum        value for the corresponding block of image sensor pixels.    -   12. The scanner system according to the preceding embodiment,        wherein the data processing system is configured for deriving        the sub-scan color for a point on a generated sub-scan based on        the surface color informations of the 2D images in the series in        which the correlation measure has its maximum value for the        corresponding block of image sensor pixels and on at least one        additional 2D image, such as a neighboring 2D image from the        series of captured 2D images.    -   13. The scanner system according to the preceding embodiment,        where the data processing system is configured for interpolating        surface color information of at least two 2D images in a series        when determining the sub-scan color, such as an interpolation of        surface color information of neighboring 2D images in a series.    -   14. The scanner system according to any of the preceding        embodiments wherein the data processing system is configured for        computing a smoothed sub-scan color for a number of points of        the sub-scan, where the computing comprises an averaging of        sub-scan colors of different points, such as a weighted        averaging of the colors of the surrounding points on the        sub-scan.    -   15. The scanner system according to any of the preceding        embodiments, where the data processing system is configured for        determining object color of a least one point of the generated        digital 3D representation of the object, such that the digital        3D representation expresses both geometry and color profile of        the object.    -   16. The scanner system according to the previous embodiment,        wherein determining the object color comprises computing a        weighted average of sub-scan color values derived for        corresponding points in overlapping sub-scans at that point of        the object surface.    -   17. The scanner system according to any the previous        embodiments, wherein the data processing system is configured        for detecting saturated pixels in the captured 2D images and for        mitigating or removing the error in the derived surface color        information or the sub-scan color caused by the pixel        saturation.    -   18. The scanner system according to the previous embodiment        wherein the error caused by the saturated pixel is mitigated or        removed by assigning a low weight to the surface color        information of the saturated pixel in the computing of the        smoothed sub-scan color and/or by assigning a low weight to the        sub-scan color computed based on the saturated pixel.    -   19. The scanner system according to any of the preceding        embodiments, wherein the data processing system is configured        for comparing the derived surface color information of sections        of the captured 2D images and/or of the generated sub-scans of        the object with predetermined color ranges for teeth and for        oral tissue, and for suppressing the red component of the        derived surface color information or sub-scan color for sections        where the color is not in one of the two predetermined color        ranges.    -   20. The scanner system according to any of the preceding        embodiments where the color image sensor comprises a color        filter array comprising at least three types of colors filters,        each allowing light in a known wavelength range, W1, W2, and W3        respectively, to propagate through the color filter.    -   21. The scanner system according to any of the preceding        embodiments where the surface geometry information is derived        from light in a selected wavelength range of the spectrum        provided by the multichromatic light source.    -   22. The scanner system according to the preceding embodiment        where the color filter array is such that its proportion of        pixels with color filters that match the selected wavelength        range of the spectrum is larger than 50%, such a wherein the        proportion equals 32/36, 60/64 or 96/100.    -   23. The scanner system according to embodiment 21 or 22, wherein        the selected wavelength range matches the W2 wavelength range.    -   24. The scanner system according to any of embodiments 21 to 23,        wherein the color filter array comprises a plurality of cells of        6×6 color filters, where the color filters in positions (2,2)        and (5,5) of each cell are of the W1 type, the color filters in        positions (2,5) and (5,2) are of the W3 type.    -   25. The scanner system according to the preceding embodiment,        where the remaining 32 color filters in the 6×6 cell are of the        W2 type.    -   26. The scanner according to the preceding embodiment where the        pattern generating element is configured to provide that the        spatial pattern comprises alternating dark and bright regions        arranged in a checkerboard pattern.    -   27. A scanner system for recording surface geometry and surface        color of an object, the scanner system comprising:        -   a multichromatic light source configured for providing a            multichromatic probe light, and        -   a color image sensor comprising an array of image sensor            pixels for capturing one or more 2D images of light received            from said object,        -   where at least for a block of said image sensor pixels, both            surface color information and surface geometry information            of a part of the object are derived at least partly from one            2D image captured by said color image sensor.    -   28. A method of recording surface geometry and surface color of        an object, the method comprising:        -   obtaining a scanner system according to any of the previous            embodiments;        -   illuminating the surface of said object with multichromatic            probe light from said multichromatic light source;        -   capturing a series of 2D images of said object using said            color image sensor; and        -   deriving both surface geometry information and surface color            information for a block of image sensor pixels at least            partly from one captured 2D image.

1. (canceled)
 2. A scanner system for recording surface geometry andsurface color of an object, wherein the scanner system comprises: anintraoral scanner having an elongated tip, wherein the scannercomprises: a multichromatic light source configured for providing aprobe light for illumination of the object, a mirror for redirecting theprobe light onto the object, a color image sensor comprising an array ofimage sensor pixels for capturing one or more 2D images of lightreceived from said object, a data processing system configured for:deriving surface geometry information from the set of captured 2D imagesand for deriving surface color information from at least one of the 2Dimages from which the surface geometry information is derived, whereinthe derived surface geometry information and surface color informationare combined to generate a sub-scan of the object, and generating adigital 3D representation expressing both surface color and surfacegeometry of the object by combining several sub-scans generated for anumber of different views of the object.
 3. The scanner system accordingto claim 2, wherein the multichromatic light source is a multi-die LEDcomprising a blue LED die and a green LED die.
 4. The scanner systemaccording to claim 2, wherein the scanner system comprises a patterngenerating element configured for incorporating a spatial pattern insaid probe light.
 5. The scanner system according to claim 4, whereinthe pattern generating element is a pattern mask.
 6. The scanner systemaccording to claim 3, wherein the spatial pattern consists of darksections and sections with light having a wavelength distributionaccording to a wavelength distribution of the multichromatic lightsource.
 7. The scanner system according to claim 3, wherein the spatialpattern is static.
 8. The scanner system according to claim 2, whereinthe image sensor comprises a color filter array.
 9. The scanner systemaccording to claim 8, wherein the color filter array is arrangedaccording to a Bayer color filter pattern.
 10. The scanner systemaccording to claim 8, wherein the color filter array comprises at leastthree types of color filters each allowing light in a known wavelengthrange, W1, W2, and W3, respectively, to propagate through the colorfilter to the image sensor pixels, thereby providing at least threecolor components.
 11. The scanner system according to claim 10, whereinthe surface geometry information is derived from light in a selectedwavelength range of a spectrum of probe light provided by themultichromatic light source.
 12. The scanner system according to claim11, where the color filter array is such that the fraction of the colorfilters in the array that match the selected wavelength range of thespectrum is larger than 50%.
 13. The scanner system according to claim2, wherein surface color information for a given surface point is atleast partially derived from the same images from which surface geometryinformation is derived.
 14. The scanner system according to claim 2,wherein the set of 2D images from which surface color information isderived from is identical to the set of 2D images from which surfacegeometry information is derived from.
 15. The scanner system accordingto claim 2, wherein deriving the surface geometry information and thesurface color information comprises calculating for several 2D images acorrelation measure between a portion of the 2D image(s) captured by theimage sensor pixels and a weight function.
 16. The scanner systemaccording to claim 2, wherein the step of combining the sub-scans of theobject comprises computing the color in each surface point of thedigital 3D representation as a weighted average of corresponding pointsin overlapping sub-scans at that surface point.
 17. The scanner systemaccording to claim 2, wherein the data processing system is configuredfor detecting saturated pixels in the captured 2D images.
 18. Thescanner system according to claim 17, wherein the data processing systemis further configured for mitigating or removing error(s) in the derivedsurface color information or the sub-scan color caused by the pixelsaturation by assigning a low weight to the surface color information ofthe saturated pixel in the computing of the averaged sub-scan colorand/or by assigning a low weight to the sub-scan color computed based onthe saturated pixel.
 19. The scanner system according to claim 2,wherein the scanner system is triangulation-based.
 20. A method ofrecording surface geometry and surface color of an object, the methodcomprising: illuminating a surface of the object with probe light from amultichromatic light source using a scanner system; capturing sets of 2Dimages of said object, wherein each set of 2D images comprises aplurality of 2D images; generating sub-scans of the object based onsurface geometry information, wherein each sub-scan comprises dataexpressing the surface geometry and surface color of the object; andcombining a number of sub-scans to generate a digital 3D representation,wherein the sub-scans are obtained from different relative positions,wherein the digital 3D representation expresses both the recordedsurface geometry and surface color of the object, wherein the surfacecolor and surface geometry are obtained concurrently.
 21. The methodaccording to claim 20, wherein the surface color of the object isdetermined by computing a weighted average of sub-scan color valuesderived for corresponding points in overlapping subscans at that pointof the object surface.